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ABSTRACT 



A seminar on direct teaching by television, held in 
Rome in December of 1966, created such interest that it was decided 
to call a second seminar, restricted to adult education, which was 
held at Sche veningen , July 1-5, 1968. The subject was the assessment 
of needs and evaluation of results. The purpose was to compare, at a 
European level, the methods already developed in these two fields, 
pool experience, and promote coordination between educational 
authorities and radio and television experts. Many kinds of research 
can be useful to the assessment of needs, but special stress should 
be put on the study of society, culture, and personality in one 
system of interdependent relations, functions, and values. Research 
into evaluation of radio and television programs must have as its aim 
to make sure that the needs met have been correctly assessed, that 
the means employed for meeting needs are the best, that targets have 
been reached, and that the operation is warranted in terms of 
cost-effectiveness. There is need for a pilot project, at national 
and international levels, on evaluation of the results of a series of 
experimental educational television programs. (EB) 
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The Council of Europe was established by ten nations on 5 May 1949, since 
when its membership has progressively increased to eighteen. Its aim is “ to achieve 
a greater unity between its Members for the purpose of safeguarding and realising 
the ideals and principles which are their common heritage and facilitating their 
economic and social progress This aim is pursued by discussion of questions 
of common concern and by agreements and common action in economic, social, 
cultural, scientific, legal and administrative matters. 

The Council for Cultural Co- operation was set up by the Committee of Ministers 
of the Council of Europe on 1 January 1962 to draw up proposals for the cultural 
policy of the Council of Europe, to co-ordinate and give effect to the overall cultural 
programme of the organisation and to allocate the resources of the Cultural Fund. 
It is assisted by three permanent committees of senior officials: for higher education 
and research, for general and technical education and for out-of-school education. 
All the member governments of the Council of Europe, together with Spain and the 
Holy See which have acceded to the European Cultural Convention, are represen- 
ted on these bodies 1 . 



In educational matters, the aim of the Council for Cultural Co-operation j 

(CCC) is to help to create conditions in which the right educational opportunities '■ 

are available to young Europeans whatever their background or level of academic 

accomplishment, and to facilitate their adjustment to changing political and social ' - 

conditions. This entails in particular a greater rationalisation of the complex educatio- 
nal process. Attention is paid to all influences bearing on the acquisition of know- j 

ledge, from home television to advanced research; from the organisation of youth i 

centres to the improvement of teacher training. The countries concerned will thereby 
be able to benefit from the experience of their neighbours in the planning and reform 
of structures, curricula and methods in all branches of education. 



Since 1963 the CCC has been publishing, in English and French, a series of 
works of genera] interest entitled “ Education in Europe ”, which record the results 
of expert studies and intergovernmental investigations conducted within the frame- 
work of its programme. A list of these publications will be found at the end of this 
volume. 



These works are being supplemented by a series of “ companion volumes ” 
of a more specialised nature, including catalogues, handbooks, bibliographies etc., 
as well as selected reports of meetings and studies on more technical subjects. These 
publications, to which the present study belongs, are listed at the end of this volume. 

| . General Editor: The Director of Education and of Cultural and Scientific 

I Affairs, Council of Europe, Strasbourg - France. 

j 

| 

|j 

li The opinions expressed in these studies are not be regarded as reflecting the 

I! Policy of individual governments or of the Committee of Ministers of the Council 

of Europe. 



Applications for reproduction and translation should be addressed to the General 
Editor. 
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INTRODUCTION 



The Council for Cultural Co-operation (CGC) of the Council of 
Europe undertakes systematic studies of new educational methods 
and techniques that can be adapted to the needs of a rapidly changing 
world. It is therefore interested in Television., a powerful instrument 
that can be used at many levels of education, especially in adult edu- 
cation , for which the Committee for Out-of-School Education of the 
CCC has launched a coherent group of activities based on new trends. 

Several European courses and seminars are devoted to educational 
tele/ision. The seminar on direct teaching by television organised in 
December 1966 at Rome by the RAI was attended by senior officials, 
educationists and producers of educational television. 

One of the main conclusions the Rome seminar arrived at was 
the unanimous view of the participants that direct teaching by television 
has to be employed as one element in conjunction with one or more other 
means of instruction , as well as in a combined operation with radio — 
without an automatic predominance of one of these elements. Educa- 
tional television — as yet in an experimental and developmental stage — 
will undoubtedly prove to be a mighty instrument in the era of “ per- 
manent education ” we are entering or have already entered, but chiefly 
as an instrument built into an integrated teaching system. 

Of special and tundamental importance for the future is the very 
interesting distinction , to be found on page 88 of the report on the Rome 
. seminar, between three types of research’. 

1. operational research, i.e. research with regard to making the 
presentation of programmes more practical; 

2. fundamental research, dealing with the relation between forms 
of presentation and the learning process; 

3. comparative research, dealing with the problem which method 
has to used in which situation. 
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The interest of the discussions decided the participants to call 
another seminar, restricted to adult education, which was held at Sche- 
veningen, 1-5 July 1968. 

Some general statements had the merit of stating several major 
problems and giving a warning against a number of illusions. They 
pointed out the usual functional fallacy of believing that methods can 
solve our problems. They are of instrumental value only; their effect 
depends on the clarity of purpose. However, the aims of adult education 
have become uncertain. Hence the urgent need for research in order 
to find the educational needs. However, it often turns out that investi- 
gations of this scope are an expensive way of postponing immediate 
action. Moreover, we may ask ourselves whether we know what we 
are looking for, and what we are going to do with the results. It then 
appears that the research for needs may help us with a clarification of 
the issues at hand, without really solving, however, the problems of 
choice and priorities. 

Adult education is said to be needed for several reasons. For 
instance, knowledge tends to be rapidly outdated in many fields. It 
is not only knowledge, however, that is needed. Skills and practical 
ingenuity, too, need revision and replacement. Changes in industry requires 
the retraining of adults. In the sphere of leisure, do-it-yourself presents 
a demand for training in various handicrafts. As to social skills, the 
developments in both industry and society at large imply the need for 
the training or retraining of adults in this respect. 

Most important of all, however, is the effect of culture change 
on the mentality of individuals. The attitudes towards others and one- 
self, while living in a mass society with few friends but an increasing 
number of strangers, is under constant pressure of revision. Our attitudes 
towards ways of life different from our own tend to be stereotyped and 
prejudiced. The stereotyped notions with which we have been brought 
up are part of our cultural heritage. They are deeply rooted, both emo- 
tionally and historically. At present, however, they are highly inadequate 
and in urgent need of revision. Our attitudes towards work in a period 
of increasing leisure, towards production in a consumer society, towards 
class and status in mobile structures — all of these are involved in pro- 
cesses of dynamic and conflicting change. We do not know yet to what 
extent adult people can really change these deep-seated feelings. How 
much insecurity can people endure without further desintegration of 
their personality? When and how do they start to create new certainties 
for themselves? And how can adult education help them with this? 

The basis for all adult education is, how can we equip adults in such 
a way that they can participate in a satisfying way in the modern socio- 
cultural system. How can adults best be motivated, informed and instruc- 
ted to occupy an acceptable and human position in culture and society? 
To answer this question we need more scientific information about our 

6 













1 



sociocultural system, for the concept of the social system is no more 
than a theoretical concept, an empty box, not filled with much empirical 
research. 

All of these considerations and questions fully justified the idea 
of holding a second seminar at Scheveningen, the subject fixed being: 
assessment of needs and evaluation of results. The purpose is to compare, 
at a European level, the methods already developed in these two fields, 
pool experience and promote co-ordination between educational autho- 
rities and radio and television experts. 
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Part I 



ASSESSMENT OF NEEDS 



The assessment of needs comes within the first phase of the research 
cycle defined at the Rome Seminar, i.e. operational research. 



A. Relationship between social research and policy-making 

The use of television for adult education must, like any field of 
human activity, entail first of all the definition of a general policy, which, 
according to Dr. van Tienen, would comprise several phases: 

I. The determination of aims chosens out of all possible aims; e.g. 
one can choose satisfying the growing demand for technical know- 
ledge, general enlightenment of the public, the removal of cultural 
bottle-necks. 

II. The designing of alternative ways to reach the aims. 

III. The designing of a specific programme. 

IV. Execution of the programme with all its problems of production 
and didactical problems. 

V. Evaluation of the programme, difficult from a technical as well 
as from a psychological point of view. 

VI. Feed-back to phases I, II, III if aims have not been attained. 

These are very general themes for social research. We must go 
further into detail and be more specific, because we note today, in many 
fields of social life, a growing need for increasingly scientific research. 
In fact, to understand what may be expected of social research , we 
must list its various aspects : 



1. Historical-sociological research, in which historical information 
and social concepts are combined. 

2. Participating field-studies. 

3. Structural-functional analysis, in which society is seen as a 
unity of the social, cultural and personality systems. 

4. Statistic^ 1 , research (opinion-polls). 

5. Panel studies — repeated statistical research with regard to 
a permanent group of respondents. 

6. Typological analysis, used to order extensive research material. 

7. Statistical experiment. 

6. Survey research. 

9. Futurological research. 

Although all the nine kinds of research mentioned above can 
be useful to the assessment of needs, special stress should be laid on 
structural-functional analysis , combined with futurological research ; 
without excluding other kinds of research, structural-functional ana- 
lysis being of basic importance for all adult education. 

In the structural-functional approach, society, culture and per- 
sonality are studied in one system of interdependent relations, functions 
and • Hues. 

OECD published in 1966 a report on “ The social sciences and the 
policies of governments ”. In this report (on page 38) three levels of 
research are distinguished: 

(a) research concerned with the general character of social change; 

( b ) research concerned with specific policy problems, e.g. health 
problems, welfare problems, problems of the aged; 

(c) research related to the practical execution of general policy. 

There is a lot of third and second level research available. Butin 
order to get more empirical information on the socio-cultural system as 
a whole we need more macro-research on the first level , combined v/ith 
projects on the second and the third levels. This would be an ideal 
situation, which unfortunately does not exist. 

The report mentions the “ Group xy8y ” in France as one of the 
first attempts at forecasting at national level a picture of the future, 
relevant to the present decision making. 

In the Netherlands the Council of Ministers has set up a committee 
to study in what way macro-social research on the first level for the 



present and the future can be institutionalised. There are similar plans 
in other countries, and there are international organisations, e.g. Man- 
kind 2000. 

In general we may conclude that macro-social research with regard 
to the socio-cultural system is necessary, but is not yet available. The 
existing research teams cannot deal with macro-social research. This 
has to be done by national and international institutes, in which the 
research teams for adult education can possibly participate in one way 
or the other. 

Although there is no systematic flow of macro-social scientific 
I information about the socio-cultural system, we can make a diagram 

I of what we ought to know (see table below). 

1 
f 

This table, or rather this kind of matrix, is simply an attempt to 
present these ideas, which does not claim to be exhaustive but 
which gives a review of the problems that arise in society and culture. 

On the horizontal axis we find all kinds of groups and categories, 
often forming sub-systems. 

On the vertical axis we find processes in which these groups and 
categories are involved. 

The task for the future will be to fill the empty squares. Some of 
the squares can possibly be filled by the research teams for adult educa- 
tion. Several participants stressed the importance of inter-disciplinary 
research groups and teams. 

The matrix does not give more than a surface description. Under- 
neath that surface the basic factors causing social change will have to 

be recognised and be brought together in a quantitative model. The \ 

matrix however contains a list of possible important factors, 

from which we should be able to say which are the most important 1 

phenomena, what are their mutual relations and how they will develop j 

in the future. 

The importance of the model was shown in the nineteen-thirties , 

in political economy by such distinguished economists as Keynes and \ 

Tinbergen. In the same way we need models covering the whole socio- j 

cultural system, whieh will tell us, for example, what is the relationship 
between : 

— delinquency and first socialisation of children; 

— the advertisement system and deviant behaviour; 

— social change and anomy; 

— social change and authoritarian personality. 
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Surface-description of the socio-cultural system 



/ - 

Groups and categories (institutionalised or not) 



sub- 


Age 


Sex 


Handi- 


Territorial 


7 •'n-territor- 


Family 


Ethnic 


systems 




capped 


groups: towns 


iai groups. 




groups 






deprived 


regions, 


Catholics, 






processes 






groups 


quarters or 


political 












districts 
of towns 


parties 

I 


I 







Deculturation 

Acculturation 

Socialisation 

Mobility 

Structural 
changes, e. g. 
urbanisation 
population 
growth 

Integration 

desintegration 

processes 

processes of 
choice (choice 
of careers) 

social 

control 

processes of 
distribution 

interaction and 
communication 

ways of expression 



Possible variables in a socio-cultural 
model 



1. Politics (power structure) 

2. Demographic factors 

3. Economic factors (production, dis- 
tribution, services, consumption) 

4. Geographical factors 

5. Social insurance system j 

6. Social welfare services 

7. Religion i 

8. Health 

9. Marriage and family 

10. Education 

11. Technology 

12. Cultural services 

13. Recreation 

14. Public finance 

15 
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value- 
orientations 

attitude- 
forming 

etc. 



One can imagine that there 
are more processes, several 
of them overlapping 
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B. Model for a functional analysis 
of the problem of adult education 

Mr. H. va;~ Praag, who was concerned with the problem of the 
assessment of needs, presented an operative model for a functional 
analysis to be carried out by various research teams. The concept of 
“ model ” he used is that of Robert K. Merton in his Social Theory 
and Social Structure (The Free Press, New York, 1967, pp. 50 et seq.: 
“ A Paradigm for Functional Analysis in Sociology ”), i.e. a methodo- 
logical pattern for sociological analysis made up of questions. The 
emphasis was laid mainly on sociology, communication theory and 
cultural pedagogy, so that the greater part of the investigation could 
be conducted by a team of specialists in these three areas. 

Mr. van Praag first of all explained a number of concepts, since 
hese are not always employed in the same manner, so that a suitable 
model could be prepared. He stressed the need not to confuse facts, 
standards and expectations. In functional analysis proper, determination 
of the cultural pattern is an important element. Models of thought are 
an essential component of this pattern; they are of primary importance 
in programming adult education. Greater knowledge of these models 
is one of the indispensable stages in the process of raising the average 
basic level of a culture, as is usually found necessary in a programme 
of adult education. This programming is possible only through continu- 
ous interaction with other education media. 



1. Subjects and targets 

If we are to achieve a functional analysis of the needs of adult 
education (with special reference to educational television) we must 
pose a number of critical questions. 

Together these questions constitute a methodological model which 
cat serve as the point of departure for a functional analysis, which 
in turn must lead to a revision of the model etc. 

In principle we follow the method of Merton. Since adult education 
via television cannot be considered divorced from permanent education 
in a broader sense, this specific aspect has only been brought forward 
in questions XIII and XIV. 

There are fourteen basic questions for our paradigm. We are postu- 
lating them here as a model for a functional analysis, taking into account 
the basic requirements elucidated. In this part of the document we have 
plotted a few outlines for a functional analysis, which, needless to say, 
are not exhaustive. 



* 

* * 
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I. On what vision of the future do we base our long-term planning 
and programming? 

II. What are relevant data for devising an operable model? 

III. To what areas ought we to confine ourselves, at least for the 
present? 

IV. What development (and/or stagnation) is occurring in these 
areas which might call for adult education? 

V. What is the actual state of knowledge and ability in these areas 
among different groups and individuals? 

VI. What may we expect of the future in this respect, and what 
aims are we pursuing? 

VII. What functions (family, cultural, social) will have to be per- 
formed in the world of tomorrow? 

VIII. How can we impart to groups and individuals the motivation 
and information they require for performing these functions? 

IX. How can the students themselves be actively involved in the 
programming of adult education? 

X. What teaching situations and forms can we create for adult 
education ? 

XI. What lags and gaps have to be made good? 

XII. What cultural patterns and models of thought have to be 
taught ? 

XIII. What special task can radio and television perform? | 

XIV. How can co-operation be achieved with other media of 
education and instruction? 



i 2. Basis for a functional analysis 

: I. In the Council of Europe our point of departure is the choice 

| of an open, pluralistic, democratic society which guarantees the liberties 

• and rights of both individual and community, and in which scientific 

; and technological media are applied to make the world habitable for 

| all nations and races. 

; n. We have to make a choice between all available data. We can 

say that generally data for our analysis can be considered relevant if 
they emanate from several sources. 

i A highly operable method appears to be the “ three sources method ” : 

| cultural history, cultural pedagogy and pragmatism. 

■ 
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It is quite feasible to collect these data by means of survey research 
(inquiries, interviews etc.). 

III. It is not possible to give a comprehensive survey of the fields 
in which adult education is needed. They can be charted in various ways. 
We have opted for a break-down into three sectors: the individual,, 
the cultural and the social sectors. 

j We shall consider these three sectors at three levels which we shall 

call sub-stratum, stratum and super-stratum, these terms being poly-inter- 
pretable. They do, however, point to a hierarchy, in which the bottom 
level is the most essential, whilst the uppermost has an ordering function. 

In the sub-stratum the structures are inter-related through irrational 
relations, whereas in the stratum and the super-stratum the rational 
! aspect dominates. 

Moreover, the stratum originates from the sub-stratum, and the 
super-stratum from the stratum. One could draw a comparison with 
a tree which has its roots embedded in the soil. The trunk above ground 
is still connected to tta roots, whereas the branches, though having the 
same basic structure as the trunk, are not in direct contact with the 
roots. In this context subterranean is symbolic of the irrational, above 
ground of the rational. 

A. Within the individual sector we distinguish the three levels: 
the Id, the Ego and the Super Ego. These terms are derived from psycho- 
analysis (Freud). If one wishes, one may also speak of subconsciousness, 
consciousness and superconsciousness (or norm consciousness). 

; B. Similarly, we can distinguish three levels in culture, often 

designated Mythos, Logos and Ratio. In the theoretical sphere they 
refer to religion, philosophy and (the branches of) science. In the prac- 
tical sphere they relate to cult, art and (the branches of) technology. 

*■ 

C. Lastly, it is also true to say that society comprises three spheres : 
community, society and state. 

Thus we arrive at the following division: 



Levels 


Individual 


Culture 


Society 


Super-stratum 


Super ego 


Ratio 


State 


Stratum 


Ego 


Logos 


Society 


Sub-stratum 


Id 


Mythos 


Community 



IV. An outline will be sketched of the various fields (religion, 
science, trade, industry etc.) showing where knowledge and learning 
are advancing and where static. 
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V. The information from point IV should be specified according 
to the current situation. 

VI. Having determined where the present-day basis of knowledge 
lies and what ceiling is needed, we must ascertain what can be expected 
and needs doing in this field. 

VII. Adult education must not be regarded merely as professional 
guidance; it is successful only if it constitutes vocational guidance in 
the full sense of the word. 

That is why every programme of adult education should also be 
viewed from this angle, and if possible supplemented with extra-vocation- 
al courses. 

A very effective division appears to be the one encountered in 
the report by the Council of Europe on “ New Trends in Adult 
Education ” (21.8.1967, page 4): 

“An inventory of aduit needs in the field of education would 
comprise the following categories: 

1. Those needs resulting from professional activities; 

2. those connected with the different extra-professional activities 
and duties existing and roles to be played within social group contexts 
such as family, local and regional settings, trade unions etc; 

3. those resulting from the desire of the individual to reorientate 
his daily life towards a meaningful cultural purport in work and leisure .” 

To have a clear general idea of the future division of functions, 
it may be profitable to use sociograms. 

Vm. The personal need in adult education can both motivate and 
stimulate. 

Conversely, a negative attitude may constitute an inertion or 
inhibiting factor. No programme of adult education can ever succeed 
without a thorough motivation survey. 

Hence the term “ interested parties ” should be construed in the 
dual sense of those who see it to be in their interests to (objectively 
interested) and those taking an interest in (subjectively interested). 

Attitude influencing may be necessary to make the objectively 
interested also subjectively interested parties. 

It is conceivable that advertising and propaganda techniques will 
have to be employed to this end. 

Investigations by American psychotechnicians and the Swiss 
Stauffer have moreover established that in the majority of cases interest 
weighs heavier that aptitude in the matter of vocational choice and re- 
training. 
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IX. There is an increasingly widespread effort to apply the prin- 
ciple of “ auto-determination ” in education. It means finding objective 
needs (Montessori). Nonetheless, this calls for a preformed pedagogical 
environment. 

X. This research problem will have to be solved by didacticians, 
sociologists, communication experts, publishers and journalists. There 
are already some publications on the subject. 

XI. This question is not only concerned with determining lags 
and gaps, but also with translating these arrears in the didactical sphere : 
“ catching up ”. 

We must first try to influence men’s attitude; an extremely difficult 
job. And even when one persuades them to make a conscious effort 
in this direction, the unconscious resistances continue to operate. 

XII. A study of the cultural patterns and of models of thought 
(e.g. basic techniques) appears to be increasingly essential for controlling 
this highly complicated information and instruction. 

This sets a special task for formal training (in logic, cybernetics, 
technology), which in this era of automation is becoming more and more 
urgent. 

Xffl. Specialists in educational television must above all ask 
themselves what contribution they can make towards this. What needs 
can television help in satisfying? To answer these questions we need 
to have a comprehensive picture of the needs of adult education. Tele- 
vision, however, has wide possibilities : it is a costly medium, and must 
be economically viable. Calculations worked out in Italy ( Telescuola ), 
France ( Telebac ), Germany ( Telekolleg ) and the United Kingdom 
(Open university) have established that a TV course usually works out 
much cheaper per head than oral instruction, provided the number 
of students is reasonably large. 

XIV. As we know from the practice of educational television, there 
has to be close co-operation between television, radio and written ins- 
truction. Other areas of co-operation have been mentioned on several 
occasions. 

We shall not study this functional division in any greater detail 
since in the main it falls outside this investigation of needs. 



C. Evolution of needs 

According to Dr. van Tienen, if we know the working of the model, 
we can establish an order of priority in the needs of adult education, 
and some of these needs can be met by direct teaching by television. 
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It is not necessary to wait until macro-social research is finished; 
we can use the existing knowledge from second or third level 
research for the assessment of needs. It will be necessary to use what C. 

Wright Wills calls “ sociological imagination ” — i.e. to predict how 
processes will develop, to formulate hypotheses and to be eager to test 
these hypotheses again and again by empirical research. 

Kahn and Wiener {Basic, Long-term, Multifold Trend, pp. 39-64) 
have given us an interesting example of how empirical facts and ima- 
gination are combined. Adult education migat be inspired by this. 

The educational needs presumed by the 1966 Rome Seminar seem 
very important and useful, but are derived from a rather narrow view 
of the modern social system. Dr. van Tienen distinguished other and 
new problems, of which he mentioned some examples: 

1. It is certain that in the future it will no longer be necessary 
for the whole working population (aged 15-65) to be engaged in the 
production process. A fairly high percentage will not be involved; 
how are they going to spend their leisure time in a way that is acceptable 
and satisfying for themselves? 

2. The changes in culture and society can only lead to progress 
if the population is able to adapt itself. Many people feel themselves 
frustrated in their old habits and get into a state of alienation and anomy. 

Is it possible, with the help of direct education by television, to change 
attitudes and provide people with a new interest in life? 

3. Our modern socio-cultural system is moving towards a situation 

which the great sociologist Karl Mannheim called fundamental demoral- 
isation: for example management participation by students and wor- 
kers. Are they able to manage universities and firms? Is there a task j 

for adult education? 

Dr. van Tienen thinks the present educational television programmes j 

for adults are very useful, but rather neutral and appreciated by what are j 

generally called the established groups in society. When, later, fuller j 

information is available about the functioning of the social system, 

it will be necessary to choose other priorities. j 



D. Preliminary survey of adult education needs 



Mr. van Praag prepared a preliminary survey of adult education 
needs, the main points of which are: 

| Our conception is based on the trichotomy: individual, culture 

and society. There is constant friction between these three spheres. 
It can be eliminated only by a clear vision of the future division of 

18 










functions between these three areas. The primary question therefore 
is how to promote: 

(a) personality forming; 

(i b ) cultural integration; 

(c) social integration. 

Viewed from the angle of the dysfunctions, this also means, how 
j are we to fight the increase in neurosis, cultural nausea and sociosis. 

The term “ cultural nausea ” originates from Freud. Sartre speaks 
of “ nausee ”, Mayrink of “ museum disease There is a general 
cultural protest, which is closely connected with the exposure and de- 
bunking of the lofty ideals of the 19th century. According to Huizinga, 
cultural pessimism is steadily increasing, partly on account of information 

entropy. 

The fight must be waged by permanent education. 

What is needed is training centres, broadcast lectures and televised 
discussions (provided these do not degenerate into negativism or into 
discussion clubs), public service centres, one or more in every town, 
which can be consulted by citizens who have individual, cultural or 
social problems. 

Most countries have already done a great deal for adult education 
in the realm of art. Co-ordination would be possible with philosophy, 
where stress should be laid as much as possible on pragmatic thinking 
in everyday life. Independent thought and creative expression must be 
promoted. We should set up studios under the supervision of artists 
and specialist teachers. 

Libraries with their information facilities are also of great importance. 

The rapid development of science and technology creates the 
very serious problem of keeping up to date. We should publish an infor- 
mation bulletin on research, make use of computers, give students inter- 
departmental information, because over the last decades the scientific 
model of thought has changed drastically. 

The traditional community groupings have lost a great deal in terms 
of safety and security (see Tonnies predicting this in the 19th century 
and Stalpers confirming it today). The evolution of our society is such 
that it calls for frequent “ re-fuelling along the road ” at “ educative 
filling stations ”. 

The building of these “ educative filling stations ” calls for planning 
of a unique kind. For that reason, the planners must continuously 
receive fresh information and instruction. At the moment we are con- 
templating four main fields of work: 

(a) Broadening the basis is necessary so as to enable a large number 
of workers to “ hold their own in society ” and to “ keep up their par- 
ticipation longer ”. 
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(b) Refresher courses should be given in technological and agrarian 
enterprises and office jobs that have lagged behind. 

( c ) Retraining is needed for workers in declining industries and 
administrations. This applies especially to younger people. 

(d) Post-graduate instruction. Nowhere is the rapid advance of 
science and technology felt so keenly as among science graduates. 

For them post-graduate instruction has become a vital necessity. Those 
primarily concerned are doctors, engineers, chemists and teachers. 

We must establish vocational guidance, ensuring not only that the 
right man is given the right job, but also that the right job is found for 
the right man. Altough this is a high-sounding ideal, we must never 
omit it as a direction-determining factor from our planning. 

With a view to realising such a programme, a permanent committee 
for adult education will have to be set up in each country, to give advice 

on co-ordination and planning. Research facilities for permanent edu- ) 

cation will have to be made available to this committee and to all other 

institutions concerned. J 

Supranational co-operation will be found necessary from the very ■ 

inception. 



E. Setting up of adult education research teams 

I. Background information 

1. The rapid development of science and technology affects society 
and culture, especially education. 

2. The idea of “ finalised studies ” is losing its meaning. Training 
for a single careeer is being replaced by life-long studying and adaptation 
to other professions. 

3. Life-long studying calls for the re-programming of compulsory 
education and adult education too. 

4. Adult education differs from education of young people. Adults 
are quite free to choose their educators. 

5. Accordingly, adults should be encouraged to go in for education 
and instruction. 

6. Adult education is becoming more and more important. 

II. Plan 

1 . All available modern educational and instructional media and 
methods should be used in the programming of adult education. 
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2. When testing these methods the entire problem of life-long 
studying should therefore be studied before deciding on the subjects 
to be dealt with by the various media, having due regard for co-operation 
between the users of those media. 

3. The educational needs of adults have to be constantly reviewed 
if a clear idea is to be obtained of those needs. 

4. Research teams are required for this purpose; teams that can 
work independently (without being tied to radio and television, for 
instance). 

5. Each research team should consist of at least three researchers 
working full time, one secretary, two assistants and two shorthand 
typists. 

6. The three researchers should be a sociologist (preferably a com- 
munications expert) a psychologist and an educationist. 

III. Duties 

1. The research team should investigate into adult education in 
the broadest sense, i.e. both individual, cultural and social education. 

2. The team should have advisers at its disposal, who could be 
consulted as a panel as well as individually. 

3. The research should be carried out in co-operation with the 
various educational institutes. 

4. Regular contact with research teams in other countries is desi- 
rable. 

5. The team should find out whether existing adult education curri- 
cula meet current needs. 

6. The team should incorporate the results of the evaluation of 
current courses in their programming. 

IV. Funds 

1. Teams should be financed by the government and financially 
supported by trade, industry and any organisation that would benefit. 

2. The money supplied by the government should come in the first 
place from the Ministries of Education and Culture, but other ministries 
(e.g. Agriculture, Health, Economic Affairs) might also participate. 

3. Trade and industry should participate to enable the average 
workers’ basic education to be broadened. 

4. In some countries, radio and television might make contributions 
from their own resources, 

5. Institutions of adult education, too, might make funds available. 

6. Lastly, trade unions, women’s associations, youth organisations 
and organisations of specialists in the most diverse fields could also 
help. 
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Part II 

EVALUATION OF RESULTS 



The concerns that became apparent at the Rome Seminar Jed 
to the evaluation of results being placed on the agenda for the Scheve- 
ningen Seminar, at which Mr. R. Quinot presented a “ Study on methods 
.< of evaluating educational radio and television programmes ”. 



A. Methodology problems raised by the different 
evaluation procedures 

Research into the evaluation of results is of the utmost importance 
to educational radio and television organisations. Its aim must be to 
make sure that the needs met by the programme have been correctly 
assessed, that the means employed for meeting these needs are indeed 
the best, that targets have been reached, and finally that the whole 
operation is warranted in terms of cost-effectiveness. 

Although straightforward at first sight, in practice these objectives 
present a challenge because of the wide diversity of factors. 

There is a major gap, when it comes to evaluating results, mili- 
tating against the expansion of radio and television, but today all orga- 
nisations have more or less appropriate instruments for measuring. 

At the outset of a methodological consideration of evaluation 
procedures there must be a description and breakdown of the methods 
actually used at the various stages, in the light of critera based on re- 
search that is less pragmatic, more objective and that makes use of 
scientific methods. 







I. Measuring programme efficiency — primarily part of overall strategy 
for launching that programme 

1. Definition of needs 

Inevitably, an understanding of what a radio or television programme 
is intended to convey is necessary before its efficiency can be rated. 

An evaluator cannot jump on to a moving train. He has his part 
to play as soon as the government, ministerial department or promotion- 
al body indicates general prospects, identifies needs and aims. 

He helps determine the manner in which radio and television aids 
are to operate: whether they are to take over, in whole or in part, a 
sector of teaching, education or advancement. 

When considering the potential aims of the educational process, 
it is important not to lose sight of the vast range of methods employed 
by the bodies responsible for putting the message across. 

Thought should be given to the possibility of combining the use 
of radio and television. It may be that it is possible to influence reception 
infrastructure — through equipment, through the information of the 
public or modification of audience make-up (re-shaping the structures 
for receiving traditional teaching, new arrangements for isolated listen- 
ers and viewers etc.). Adjuncts must not be neglected: companion 
literature, correspondence courses, recordings making use of other aids,., 
such as records, films, sound tapes, combined sound and picture tapes, 
and later on EVR (electronic video recording) and programming/ 
response on computer terminals, operating under the group interroga- 
tion centre or team sharing system. 

Clearly there can be no such thing here as a single evaluator: 
for this reason the term is used in a generic or plural sense. The evaluator 
is multiple from the outset. When the final report comes to be presented, 
he will be legion. 

2. Intake infrastructure 

It must be decided how the message is to be put across, i.e. through 
regular networks or not (closed circuit). Is the audience to be reached 
through the automatic and single-direction means of a transmitting 
station operating on a micro-wave network or by cable, or through the 
more indirect and selective means of aids — chosen with definite aims 
in view: optically reduced prints (COMOPT), tape recordings etc.? 
The aids used have an unmistakable influence on the scope and form 
of audio-visual messages. Appreciable differences are to be found 
at the utilisation stage, from the point of view of educational practices 
and of effectiveness. Before any research can be carried out into effi- 
ciency, it is essential to have precise details about the receiving equipment 
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available, especially its technical condition. A preliminary analysis 
of results implies a breakdown of audience infrastructure (number of 
sets, pupil/set ratios, equipment utilisation rate). The rate of increase 
in sets, unlike other consumer goods, is not directly indicative of 
educationists’ faith in the medium. Card indexes giving full details may 
well be adequate, depending on the scale of the undertaking. It is neces- 
sary in most cases to resort to statistical methods in conjunction with 
powerful data-processing equipment, when determining the predomi- 
nant characteristics of the audience in relation to equipment utilisation. 

3. Attitudes 

Studies have been carried out from the psychological and sociolo- 
gical viewpoint into attitudes, trends and resistances among the public 
(teachers and students), but in most cases they have merely recorded 
the bare facts. Such attitudes can admittedly be left to evolve of their 
own accord — or an attempt can be made to speed the process by a 
variety of interventions. 

Such interventions have invariably provided confirmation of 
the complex nature of rejection or acceptance processes. The reactions 
under consideration no longer hinge exclusively on the medium, being 
inherent in the different stages of a process of change entailing its own 
facts and figures, regardless of the media used. 

The evaluator agrees, by integrating these extraneous facts and 
figures, to dovetail his research into effects within the more general 
framework of the purpose of education — which is already to some 
extent reflected in the attitudes of the teacher. 

Before reaching the stage of assessing the effects expected to follow 
the elaboration of the message, and its translation into palpable terms, 
it is important to identify the scope for a psycho-sociological survey 
of the transformation of the educative act, of the relationship between 
the teacher and the student, and finally the process of gearing the pro- 
gramme to the external facts relating to basic objectives. 

4. Defining desired effects of audio-visual message 

The audio-visual message can be regarded as a computer product.. 
An overall strategy for launching the product on the market has gradually 
been evolved, through an analysis of demand, familiarity with audience 
infrastructure, and consumer attitudes. 

The operational models presented recently, especially for direct- 
teaching television programmes, have taken an increasingly important 
share in this operational stage. Systematised programmes are taking 
the place of “ self-service ” programmes (learning a foreign language, 
preparing for an examination, moving from one grade to another 
within a job. further training in science etc.). 
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In such cases the evaluator sees precise goals — usually with se- 
parate stages and times allocated. This does not apply to programmes 
designed for “ cultural enrichment ” purposes, so that his conclusions 
here are bound to be much less meaningful. (Intervention by an evaluator 
may even be said to be useless in these circumstances.) 

These comforting priorities are thrown out of kilter when the 
programme comes to be devised. Teaching art and technique intermingle 
to produce a message, through a series of complex operations which 
add new and for the most part unpredictable creative elements to the 
objective data relating to the proposed production: however precise 
the design behind an audio-visual message, the finished product will 
have an originality of its own, which in some cases will be surprising. 

An overall or sequential analysis of a broadcast makes it possible 
to identify targets, means used for achieving them, and the results 
anticipated. It may involve analysing a situation or object, providing 
initiation into behaviour patterns, developing a skill, supervising the 
acquisition of a concept, a term, an ability, or a thought process. 

Systematic analysis reveals the logical pattern represented by 
exposition of the subject, questioning, checking of answers, individual 
and group exercises. 



In place of intentions, the analysis grid provides a methodological 
approach to a list of the effects anticipated from the message as outlined. 
Measurement of the effects, both quantitative and qualitative, is then 
a. matter of varying simplicity. 



At operation point, further effects will appear, and far from re- 
presenting the residual content of a survey of results, these will provide 
essential information. They may be connected with the psychology 
and physiology of perception, or with the behaviour of the subject 
(emotional, intellectual and social progress). 



II. Methods of evaluating results 



1. Documents enabling immediate reactions to be speedily identified 
and hypotheses formulated 

This is the method of analysing letters from listeners and viewers, 
sometimes encouraged by questions in the form of assessment cards, 
combining open-end and closed questions, advocated at the moment 
by several organisations. These are submitted by the user immediately 
after the broadcast, and permit a rapid feed-back of considerable value 
to producers who are endeavouring to re-devise a programme or to 
prepare a new series. 
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The next stage is the survey by means of standard questionnaires, 
providing substantial amounts of quantitative information. In the case 
of isolated students, adolescents and adults — whether or not re-grouped 
— it provides a highly flexible instrument 

The evaluator must however entertain considerable reservations 
regarding these documents. Little or nothing is known about the compil- 
er of the answers and it is impossible to assess the yardsticks by which 
he operates. These disadvantages are countered by using a sample 
group of informers who can be sought out and assembled. A quantitative 
drop in the supply of information is then offset by a qualitative increase. 

2. Systematic observations 

As it is clearly difficult to create genuinely experimental situations, 
the clinical method of observation is frequently used in studying effects 
and results. 

This entails straightforward observation of the subject while the 
message is actually being conveyed, noting built-in attitudes and verbal 
reactions. Actual visual and sound techniques are used for recording 
reactions. 

In conjunction with measuring instruments (tests that have already 
proved valid in familiar educational situations), the clinical method 
makes it possible by means of a thorough-going study of the facts to 
measure results and to pinpoint the main effects. 

Generalised comparative observation also makes use of measuring 
instruments (with often new equipment) for systematic research among 
comparable sections of the population. 

The aim of careful research of this kind, through lines of reasoning, 
is to indicate the significant differences in results obtained by the different 
groups being compared. The actual selection of subjects and of evaluation 
processes, the formulation of hypotheses and the determination of 
variable factors, all make it possible to move on from the stage of 
probabilities, and to equip educationists, technicians and planners, 
who are also anxious to promote effective programmes. 

Of course, it is not always possible to know what is to be measured, 
or to have the necessary instruments to hand. Statistics already used 
for audience research and programme ratings are essential for analysing 
results. In this way assumptions can be verified, correlations between 
measuring samples determined, and each factor assigned an absolute 
value and a relative value (factorial analysis and variation analysis). 

3. Experimenting 

One essential difficulty is encountered in attempting to apply a 
strictly experimental method. Specialists are asked to evaluate a program- 
me already in existence or being prepared. It has not been devised for 



experimental purposes, its aims prove to conflict, and its targets are 
indeterminate. The whole range of technical processes has been tapped 
without distinction and without objective justification. 

Whenever it is possible for research workers to co-operate with 
producers from the outset, however, if not for the full treatment of the 
programme at least in respect of one sequence both coherent and signi- 
ficant, conclusive results can be provided by the experimental method. 

To date no organisation appears to have systematised a simulation 
procedure of this kind with test programmes or mock-ups. Firstly, 
there is considerable confusion at that level between pure research, 
applied research and the pre-testing of programme models intended to 
become operational through mass production. Besides this, finance 
is a major problem, as it is a costly business to design mock-ups and 
experiment with them, and then to effect the necessary research. The 
crux of the matter, however, lies in a state of mind rather than in finance. 
It is proving hard for educational radio and television to move on from 
the creative, spontaneous and pre-industrial stage. The proportion of 
investments and operating expenditure devoted in numerous European 
programmes put on by progressive organisations to feed-back and 
experimental testing varies between 0 % and 5 % of the total budget. 
Present thinking is that the proportion connected with previous studies 
and procedures relating to programme launching should account for 
25 % of the budget. 

4. Contribution of new scientific methods 

Initially audio-visual messages were classified in terms of the parti- 
cular aid used, but this is no longer the case. The change-over from one 
form of aid to another is not now determined simply by profitability 
demarcation lines. A television programme may use Super 8 mm film 
plus ten other kinds of picture, be put out on double track (SEPMAG), 
reproduced by 16 mm optically reduced print (COMOPT) or on video- 
recorder tape. Reception appliances using cassettes (EVR type) do still 
more to promote aid flexibility, incidentally disposing of the disadvant- 
ages attached to programme grids and enabling the consumer to have 
access to the product whenever he wishes. 

The education authorities, and broadcasting organisations also, 
are strangely hesitant when it comes to technological developments. 
The writer’s terms of reference do not take in this point, but there is 
no ignoring an apparent readiness to stay in the first generation of 
radio and television aids — notwithstanding the existence of multi- 
purpose machinery, group questioning consoles, and computers with 
terminals ready to respond to programming techniques. 

Research into results is made easier by integrating programmed 
instruction techniques into the initial elaboration stages. Prior standards 
are established through the definition of targets, sequence-cutting and 
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progress charts, and the discipline of these standards balances or per- 
haps even directs the unpredictably creative achievements of the three- 
headed (educational, artistic and technical) Cerberus. 

Attempts have been made to find ways of combining programmed 
instruction and audio-visual message production methods. In a few 
quarters complete integration has been advocated, while in others the 
suggestion is that they should be used to complement each other (pro- 
grammed written exercises, use of straightforward individual machines). 
In all cases it has proved easier to measure results — because the initial 
aim was precisely to achieve a result. 

Similar advantages are to be derived from group questioning 
centres. The results of questioning can be immediately gauged, as they 
are geared to a system of questioning which in turn is graded in terms of the 
expected replies. 

Pupil participation is a foregone conclusion. In this way behaviour 
patterns are established that are properly adapted to die aims of education. 
Armed with clear-cut criteria of this kind, the evaluator is able to reach 
meaningful conclusions about the effect of the programme. 



III. Evaluation and effectiveness 



So far no significant attempt has been made in Europe at an evalu- 
ation in economic terms that examines cost-effectiveness. 

At the third Congress of the European Broadcasting Union (March 
1967) on educational radio and television programmes, the Rapporteur 
for Committee II, on “ Organisation and Planning ”, commented that 
a start had been made on team work between economists, educationists, 
and persons engaged in communication, with a view to agreeing on a 
general approach to problems posed by the economics of educational 
radio and television. It had been pointed out in discussions that these 
issues were extremely complex and that comparative studies presented 
considerable difficulties because of the wide range of costs and expen- 
diture that could be charged to one budget or another. 

So far the use of audio-visual media has not been posed in terms 
of effectiveness — no doubt because only a marginal proportion of the 
funds available for educational purposes is devoted to such use. 

It was demonstrated in a survey effected by the International 
Institute for Educational Planning, covering 21 countries using radio 
and television in education, that the budget for these programmes totalled 
75 million francs — aproximately l/240th of the budget of the French 
Ministry of National Education alone. 
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1. Concept of efficiency 

Objectives must be sufficiently clear for each programme to be 
allotted a purpose that corresponds to basic needs. 

Such objectives (examination, moving from a given degree of 
skill to a higher one, initiation into a language in the case of direct 
teaching for instance, and other more devious objectives in the case 
of “ cultural enrichment ” courses) are to be attained through the inter- 
mediary of regular intake of the product, and a comparison of results 
and intentions. 

At such a level it is of little significance whether introducing one 
new communication medium is or is not more effective than another. 
The evaluator is merely concerned with identifying effects and assessing 
results. At first he shows whether or not the objectives have been reached. 
If his yardsticks for measuring efficiency are quantitative (audience survey 
and rating) and qualitative (as revealed by a study of feed-back, clinical 
observation and advance experiments or pretesting) at the same time, 
they are designed essentially to confer “ authority ” on the programme, 
i.e. to authenticate transmissions of the programme and to measure 
the degree of its success. 

2. Concept of effectiveness 

Once trie programme has indicated its efficiency through results, 
thought must be given to whether it shows a return. 

What circumstances produce the best results ? What are the 
media to be used (in isolation or combined, with or without other infor- 
mation aids)? 

The lower the cost of production compared with the results obtained 
for an individual or collection of individuals, the greater the return 
shown by the programme. 

The return shown by radio and television aids depends on the 
productive ensemble of the educational system. A first-rate programme 
of proven efficiency is not an end in itself. It is dovetailed within the 
planning of educational development, of which it is indeed one of the 
means (in some cases the only means). 

When the evaluator comes to assess returns, he has at his disposal 
the same statistical information (investments and operation) as that 
produced by those engaged in the direction of education (number of 
pupils, teachers, establishments, examination results, duration of atten- 
dance at school, transfer from one level to another, equipment rates etc.). 
He also needs statistical information from producing entres (education- 
al elaboration, technical elaboration, different stations for production, 
broadcasting, equipment, information and training in individual or 
group reception). 
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If we accept the diversity of aims, the range of variables and the 
need for quantitative and qualitative measures, we shall also agree that 
a return must be assessed first and foremost in monetary terms (the only 
terms carrying any weight with planners and the authorities), but also 
m other terms — provided they are probative, comparable and con- 
ducive to generalisations. 

IV. Side issues of evaluation 

t 

1. Evaluation = team spirit 

Evaluation is a permanent process, occurring at all stages (from the 
preliminary survey to the final summing-up) throughout operational 
planning. Evaluation consists of teams undergoing constant renewal* 
it is also a state of mind. ' 

Evaluation teams intervene at different stages, in different forms. 
It has been shown by the difficulty of hitting on common preoccupations, 
methods and terminology that it is not simply to bring together a set of 
skills; colleagues must be found who can see further than their own 
speciality und thus archieve some degree of versatility. The 
hostility of educationists and of visual and sound technicians to methods 
of measuring efficiency and calculating effectiveness will diminish as 
they become uncreasingly involved in the whole process of evaluation. 
The converse is obvious. The evaluator cannot establish criteria and 
make choices without extensive familiarity with the dual experience 
acquired by the audio-visual specialist, and of the manifold implications 
of this sector of education. 

2. Integration of evaluation results and their general application 

The processing of all the information collected is essential, and 
demands considerable resources. Evaluation is a close neighbour to 
applied research, and has many points of contact with pure research. 

Studies may be carried out by producer organisations or bodies 
working in conjunction, university institutes or research centres associat- 
ed with groups representing business or social interests. 

Evaluation investments, more or less distinct from research invest- 
ments, must correspond to budget lines that become clearer as the tasks 
to be effected become more specific. 

When the work, effort and finance devoted to a deliberate policy 
of rationalising undertakings are on this kind of scale, radio and television 
organisations are bound to take the fullest account of the evaluators’ 
findings. In some cases such findings may lead to painful adjustments, 
but they will invariably promote a greater use of radio and television 
m education — according with the general objectives of education, 
which after all is the aim of research. 



Still the most usual situation is post facto evaluation. Decisions 
were reached a long time ago, programme production is under way. 
There is no breaking loose from the system: in many cases it will be 
years before the results of evaluation can be taken into account, and the 
necessary changes effected. 

Experimental programmes are the best solution. From the outset 
the evaluator is on the shop floor. When it is impossible for a mock-up 
to be produced, or for a programme to be pre-tested, those in charge 
must see to it that the evaluator has a chance to intervene at the very 
start of the asseraby-line, i.e. at the design stage. 

The attitudes of the various participants in the overall task are not 
yet cut-and-dried at this stage. Confrontation is a possibility, a vast 
range of assumptions can be formulated, and conclusions are something 
for the future. 

The results of evaluation make it possible to move on from the 
test stage to the general production stage with considerable technical 
and financial “ secuiity ”. Educationists, technicians and users all stand 
to gain from this procedure, since it leads to a system of planning that 
matches means to aims, the product to demand; it also replaces a prag- 
matic approach by a rational approach, enabling both the creator of 
the audio-visual message and the consumer (teacher and pupil) to 
contribute with maximum efficiency. 



B. A significant experiment : TELE AC 



Several national experiments were presented at Scheveningen. 
This confirmed the variety of methods and the difficulties of applying 
them, but was nervertlieless of great practical interest. 

One of these experiments at evaluation was particularly significant: 
that of the Netherlands TELEAC Foundation, which since 1965 has 
been making educational television broadcasts of five types: courses on 
given themes, refresher courses, inter-faculty courses, art courses and 
language courses. 

For research, TELEAC, which does not have a complete department, 
collaborates closely with a research institute of the University of Amster- 
dam and with the research department of the Netherlands Television 
Foundation at Hilversuin. At present research is limited to three fields: 
(a) exploration of the market, ( b ) group testing, and ( c ) evaluation. 
These three types of research are made for every course. 

The evaluation method has two parts ; it comprises group testing and 
scientific research. 
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Group testing can be done before, during and after the production. 
A distinction is therefore made between pre-testing, simultaneous 
testing and post-testing. 

When a suggestion and the idea for a series have been made ready 
for production, one or two lessons are chosen according to standard 
criteria and are worked out into specimen lessons; the ideal, of 
course, is to make two different lessons, so that the groups that see 
them really have a choice. As a rule, two different kinds of groups are 
invited: on the one hand potential viewers and on the other as large 
a variety of experts as possible. Each group contains 15 to 25 persons, 
recruited from a radius of 30 miles around Delft. 

This geographical condition, and the opportunity to criticise 
TELEAC in its own studio, encourage the public to come. What all 
these people have in common is the willingness and interest to attend 
some course or other during their spare time. 

Another resource open to TELEAC is to make an appeal to listeners 
to some course already broadcast on the subject under consideration. 
For example, twenty former listeners to the course in the “ Logic ” series 
were asked to pre-test a series on new scientific models. In the case of 
music courses some so-called “ average ” choirs were invited. 

When the specimen lesson has been broadcast, questionnaires 
are submitted to the groups, who answer open and pre-coded questions 
on the content, form and presentation of the lesson. This is the written 
phase of the test. Later, a discussion is held: this is the oral phase. 
We should note that the meeting of potential course-members takes 
place before the meeting of experts: this latter group has a permanent 
nucleus formed of a didactician, a teaching psychologist and a member 
of the Institute for Perception Research. 

The second phase of the evaluation is based on scientific research. 
The problem is to know who have watched the programmes, the catego- 
ries being according to age, sex, education, profession, number of 
programmes watched, and the opinion of those concerned on the form, 
content and presentation of the course. 

This research method provides much useful information as to the 
constitution of groups of course-members and viewers according to 
a number of characteristics, their actual viewing behaviour and their 
opinioi -— mostly pre-coded — on certain aspects of the course. In 
this way it is possible to measure the effectiveness of the series and the 
extent to which one becomes aware of increased knowledge, as well 
as the way in which one sets about looking for further information. 

The research department of the Netherlands Television Foundation 
is at present engaged in a continuous enquiry into the viewing and 
listening behaviour of people of the Netherlands aged 1 5 and over. 

33 



I'lirritriTififMiirTi-r 






A group totalling some 1,500 people has been formed which, 
being carefully selected, seems representative of Netherlands television 
viewers. 



C. General discussion 



A general discussion was held at Scheveningen on the report by 
Mr. R. Quinot. The delegates expressed approval of the report as a whole. 

In all cases (even those of countries that had not yet been able to 
start evaluation) the delegates testified to the constant concern felt 
by adult education organisers in this field, emphasising the following 
aspects: 

Impact evaluation is made possible by the functional analysis of 
needs, which determines objectives and the area within which program- 
mes are to operate. The programme provides one or more satisfactory 
answers to the needs (explicit or implicit) of a given group of adults. 
These answers may lead to a re-thinking of the usefulness of the program- 
me, and of the kind of message it puts across and the form it takes. 

The part played by radio and television in adult education covers 
fields which defy traditional rules: it is therefore necessary to have 
new planning procedures. 

Those responsible for programmes, and educationists, are acutely 
aware of the economic, social and cultural consequences of what they 
are doing, so that they are not disposed to go into the extremely 
sophisticated findings of forma’ research. Their hope is that an analysis 
of results will mean that they can regain a sense of perspective, work 
out new hypotheses, find theoretical and practical means of improving 
messages, and finally plan a strategy for putting across these messages. 

Result evaluation thus forms a part of a line of educational research 
that is both forward-looking and operational. It accords with the inten- 
tions that are basic to adult education, as confirmed by the seminars and 
study groups arranged by the Committee for Out-of-School Education. 

Before any programme for adults can be set up, a plan for research 
into effectiveness must be devised. 

Because their interests appear to conflict with those of the planners, 
educationists are still at times afraid that impact research is merely 
a means employed by the planners, a manifestation of their overbearing 
methods. Actually, the objective is a shared objective: to see what pro- 
grammes and methods will produce the best results, on the basis of 
quantitative and qualitative information. 



The comments of several experts who presented reports on national 
research, and the requests voiced by representatives of bodies not yet 
able to carry out evaluation, underscored the vital need to exchange 
the main findings of programme result analysis throughout Europe. 

Thinking to date on planning the joint use of new educational 
media is not yet sufficiently well-defined, and not enough statistical 
data are available, for those taking part in the seminar to be able 
to give a more precise answer to the effectiveness concept raised by the 
Rapporteur. 

The next seminar (Sweden - June 1969), on the problems of co- 
operation between radio and television organisations, educational 
establishments and government authorities, should enable further light 
to be shed on the contribution to be made by impact analysis in this 
field. 



D. Outline for a pilot project on the evaluation 
of the results of a series of experimental educational television 

programmes 

presented by Mr. Quinot 

A number of methodology difficulties have been revealed by theoret- 
ical studies on the methods of evaluating educational radio and tele- 
vision programmes. Such difficulties have been confirmed by a careful 
analysis of the reports on results obtained by various national organisations. 

These studies have led to conclusions which, although interesting, 
are frequently not particularly meaningful and scarcely ever capable 
of general application. 

Over the last five years there has been considerable progress in 
this field throughout Europe and it is now possible to look to highly 
competent specialists, having at their command an increasingly sophisti- 
cated battery of equipment. All organisations also want to indicate with 
greater clarity what action is to be undertaken, and constantly to improve 
their output, which in turn will be made more effective by a methodical 
evaluation of effects and impact. 

This urgent need should induce the member states of the CCC 
to take part in a carefully planned and elaborate scheme to study the 
most characteristic aspects of impact evaluation in a real educational 
situation, to draw meaningful conclusions and decide what action it is 
appropriate to recommend. 



* 

* * 
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The pilot scheme might take the following form: 

1. Objective 

1.1. At national level 

Application in a real educational situation of various methods 
of impact analysis especially geared to the needs of the public, the 
intentions of educationists and the general purpose behind the pro- 
gramme — as a part of the continuous development of combined tea- 
ching methods. 

(a) Constitution of the team and analysis of duties (who? when? 
why? how?) throughout the scheme up to the analysis of results, and 
the resultant recommendations, both theoretical and practical. 

Teachers and vision/sound technicians become less suspicious 
of the methods whereby efficiency and effectiveness are measured as 
they become increasingly involved in evaluation processes as a whole. 
Conversely, evaluators are in no position to lay down criteria, devise 
experimental plans, instructions on how checks are to be performed, 
suitable material and tests without the creators of the audio-visual 
method and joint material taking part on a permanent basis. 

(b) Selection of evaluation method or methods, reasons for this 
selection, devising material for checks, ranging from the rudimentary 
to the sophisticated. 

Application to the population as a whole or special population 
samples. Scientific analysis of experimental process, its limits and pro- 
blems. 

(c) Analysing impact from the following points of view: 

— needs, 

— aims, 

— scope of audio-visual methods, and form assumed, 

— efficiency. 

(d) Statement of findings and recommendations, so that it is possible 
to move on from the trial stage to one where producers can look to 
general standards for production, in conditions that ensure educational, 
technical and financial stability. 

1.2. At international level 

A comparison between several European countries of the way 
in which the pilot scheme is conducted, methods used, arrangements 
for applying such methods and of their effectiveness, should help pro- 
moters, administrators, educationists and planners to determine the 



future framework for a logical use of joint educational media likely 
to match the needs of adults in the field of training and advancement. 



2. Theme 

Selection of an area geared to the needs generally voiced by the 
majority of member countries. (How to effect a functional analysis of 
educational needs?) 

According to reports recently published by the CCC, The use of 
television in adult education — European achievements (Mrs. M.G. Puglisi) 
and New trends in adult education (Committee for Out-of-School Educa- 
tion, August 1967), the most popular choice is likely to be an introduc- 
tion to economics. 

Within this area, one theme would be taken corresponding to 
a coherent educational ensemble of knowledge, thought and behaviour 
patterns. Examples would be: circulation of goods, wages, economic 
crises, means of production, company management etc. 

The choice of one area, and then of a single theme, geared to national 
interests and the general direction of each organisation’s planning, 
will raise the pilot project to its proper status of comparative survey, 
as soon as a range of common educational aims can also be defined. 

Far from being regarded as impediments to an ideal experimental 
situation, national preoccupations, users’ basic concepts and attitudes, 
and the various social and economic realities constitute the essential 
data for this European experiment, and it is essential that they shall 
be brought well to the fore. 



3. Number of programmes 

Quarter-hour, twenty-minute or half-hour units will be adopted 
after enquiries into national broadcasting standards. The minimum 
for the experimental series should be three twenty-minute broadcasts, 
but it would be better if there could be five. Depending on the subjects 
dealt with, the broadcasts could be spread out over three weeks. 

4. Formation of European study group 

The study group would be made up as follows: 

(a) Officially, of a minimum of four or a maximum of eight Euro- 
pean experts, responsible for carrying out surveys before the project 
gets under way, the co-ordination of research, comparing methods and 
results and summarising conclusions; 

( b ) Unofficially, of all the national teams taking part in the scheme. 
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■ Modelled perhaps on the teams covering • a range of subjects as 
suggested for demand analysis by the Scheveningen Seminar, their 
membership varying according to the practical details noted in the 
previous report on impact assessment, the national teams represent the 
foundation for the experiment. Teachers and technicians will have a 
part to play at every stage. Not concerned with the previous selection 
of the area, theme and objectives, these teams conduct the experiment 
on the social and economic, educational, cultural and scientific bases 
peculiar to themselves. The actual description of the original way in 
which the experiment is conducted should be one of the most valuable 
contributions that the comparative survey has to make. 

During the time that the experiment lasts (30 months, see following 
paragraph), the study group might meet two or three times at the Council 
of Europe headquarters, Strabourg, and at least once in each partici- 
pating country (in a place selected by that country), so as to give on-the- 
spot consideration, with the national teams, to the way in which the 
exercise is proceeding. 

5. Timetable 

Once the project has been adopted and the Council of Europe 
notified of ten or so member countries disposed to take part, the 
following timetable, extending over 30 months (but possibly to be 
cut to 24) and broken down into stages averaging 6 months, might 
be adopted. 

(a) Preliminary stage (first to sixth month) 

— Advance survey of national requirements, main needs in adult 
education and production standards. 

— Comparison of national spheres having priority, selection of 
one. Examination of theme likely to be chosen for experiment and its 
educational objectives by appropriate bodies consisting of educationists 
and users. Choice of common theme and restricted range of aims. 

— Survey of existing national productions in the field under 
consideration and standards concerning broadcasting. 

— Formation of national teams and agreement between the CCC 
and member states on how exactly they are to take part in the scheme. 

(b) Stage at which the programmes as a whole are devised (seventh 
to eleventh month) 

— Drawing up broadcasting plans, devising a combination of various 
aids. Choice of methods, preparing the experimental plan and evaluation 
instruments. 

— Comparing projects, so that assumptions may be adjusted, 
common aims re-thought if need be, and a plan for a comparative study 
adopted. 
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(c) S/age at which all the programmes are produced (eleventh to 
eighteenth month) 

— Technical production. 

— Advance experiments in accordance with method selected. 
— Analysis of potential audience, preparing samples. 

— Comparison between original projects and programmes for 
further adjustment of objectives and the plan for a comparative study 
if necessary. 



(d) Application stage (nineteenth to twenty-fourth month) 

— Putting out programmes. 

— Application of evaluation machinery. 

— Impact analysis. 



— Each national team to draw up report on the experiment plus 
a detailed description of the way in which the exercise has proceeded. 



(e) Concluding stage (twenty-fifth to thirtieth month) 

— Comparative study of national reports. Group of experts to 
draw up summarising report indicating directions opened up by syste- 
matic analysis of impact, in connection with future action, improving 
productions, means of continuing to satisfy requirements. 



* 

* * 



1. The theme selected must possess genuine attractions for all 
the states taking part. From this point of view, economics is a suitable 
subject. Account is to be taken of the forward plans already devised by 
the member states’ radio and television organisations (quest for a ioint 
line of action). 



2. Over the last three years, out-of-schooi education experts have 
repeatedly pointed to the advantages of combined teaching methods. 
These countries able to do so will derive advantage from sending out 
companion literature in connection with the programmes. The research 
carried out should bear in mind the use made of aids of this kind. 



3. An equal number of programmes, lasting exactly the same time, 
must be taken so that comparison is facilitated. The choice of number 
(three, five or seven) and of programme length (15, 20 or 30 minutes) 
will depend on the prior study of the appropriate organisations’ normal 
planning standards. The organisations might in fact incorporate the 
programmes into wider series, and research could be confined to the 
experimental programmes. 



4. More countries will be likely to take part in research if certain 
arrangements are made between countries where the same language is 
spoken, for production purposes at least (e.g. : Dutch-speaking Belgium 
and the Netherlands, Walloon Belgium, French-speaking parts of 
Switzerland and France). 
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CONCLUSION 



A critical approach is necessary to evaluate the possibilities of 
adult education. Mr. van Praag very rightly concluded “ Let us beware 
of wishful thinking and over-hopeful acting. ” 

Another danger is that of over-estimating methods, which are mere 
instruments. 

Research must not be an expensive way of postponing the solution 
of problems. It must be supplemented by experiments. 

This research must be done in an atmosphere of international 
contact. Some pilot projects could be carried out by several countries 
working together. 

To compare the evaluation of our results, we need a common 
language — if possible, a simple and comprehensible one. We must 
not be overcome by the magic of words, or by a pseudo-scientific logo- 
machy! 

With this common language, research can be done on an interna- 
tional level. The Council of Europe can help to co-ordinate these acti- 
vities. 

This second seminar at Scheveningen has shown how far we still 
have to go. Almost everything remains to be done, both in assessing 
needs and in evaluating results. 
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